Quicksort leave-pair-out cross-validation for ROC curve analysis

Abstract Receiver Operating Characteristic (ROC) curve analysis and area under the ROC (AUC) are commonly used performance measures in diagnostic systems. In this work, we assume a setting, where classifier is inferred from multivariate data to predict outcome for new cases. Cross-validation resampling method estimating prediction of on not inferring it. Tournament leave-pair-out (TLPO) cross-validation has been shown be better than other methods at producing ranking that can curves areas them. However, time complexity TLPOCV, $$O\left( n^2\right)$$ O n 2 , means it impractical many applications. article, called quicksort (QLPOCV) presented order decrease obtaining reliable n\log n\right)$$ log . The proposed compared with existing ones an experimental study, demonstrating terms AUC values QLPOCV produces as accurate estimation outperforming both k -fold leave-one-out cross-validation.